feat: #44 deploy the server build to where crate-api runs it #45

erikhfj · 2026-10-10T19:35:25Z

Why

rachelks on #42: "workflowen bygger den nye serveren, og restarter crate-api.service, men fant ikke noe steg som deployer selve backend-builden".

She is right. crate-api.service runs node dist/index.js from ~/T19-Project-2/server. The workflow built server/ in the runner's own workspace, _work/T19-Project-2/T19-Project-2, and never copied that build anywhere. Restarting the unit just brought back whatever process had been installed on the machine by hand, which is still an early prototype from 23 September.

What this does

The deploy step now goes through ~/T19-Project-2, which is a git clone of main as of this issue:

git checkout --force $GITHUB_SHA there
pnpm install and build server/
systemctl restart crate-api, poll until /graphql answers
run a smoke suite through the Apache proxy the browser uses

Checking out the tested SHA rather than rsyncing files means what runs is exactly what CI built.

The VM work behind it

~/T19-Project-2 was a hand copy with no .git. Replacing it needed git access from the machine, which it did not have:

generated ~/.ssh/id_ed25519_git on the VM
added it to the git.ntnu.no account (gh api user/keys, id 3852)
ssh-keyscan for the host key and an ~/.ssh/config entry so git picks the key up
cloned main, restored .env

.env is in .gitignore, so git checkout --force leaves it alone. Verified: same md5 before and after.

Three bugs from earlier that this does not repeat


`grep -q` + `pipefail`	closed the pipe at the first match, `systemctl` died of SIGPIPE, and the `if` took the else branch and skipped a service that was there. Now `[ -f /etc/systemd/system/crate-api.service ]`.
`sudo` with no TTY	"a terminal is required to authenticate". Needs a NOPASSWD rule scoped to three `systemctl` commands, already installed on the VM.
`sleep 2` before the health check	`Type=simple` reports active before Node binds the port. Measured 1878 / 1850 / 1864 ms, so `sleep 2` left 120 ms of margin and `/graphql` returned 503 through Apache. Now polls.

What was tested

Against the live VM:

Check	Result
`~/T19-Project-2/.git` exists	yes, HEAD `0fbd54a`
`git fetch origin` with the deploy key	ok
`git checkout --force HEAD` keeps `.env`	md5 identical before and after
skip guard when `server/` is absent	correct, exits 0 with the reason
`sudo -n systemctl is-active crate-api`	`active`, no prompt
`POST /graphql`, `GET /project2/`	200 both

Also pnpm format:check, pnpm lint, pnpm typecheck and pnpm test all pass.

What still waits on #42

server/ is not on main yet, so the step's skip guard is what runs today. Once #42 merges, the next push to main deploys the API end to end and the smoke suite runs for real. That is the last acceptance line of #44:

a reboot of the VM leaves a working API

I cannot verify that until there is a server to run.

Closes

Implements #44. The reboot acceptance line closes when #42 has landed and a deploy has run against it.

rachelks · 2026-10-10T21:09:58Z

Looks good overall! Just two small things before merging:

Could you update the branch with the latest main, since the source-map-js fix from #43 has already been merged?

Also, the GraphQL smoke tests currently seem to check HTTP status codes only. Since GraphQL can return 200 even when the response contains errors, could we also check for an errors field in the response?

erikhfj · 2026-10-12T19:43:33Z

Both fixed, and the second one was demonstrable immediately.

Branch updated. Rebased onto main so it now carries the source-map-js fix from #43. 8a97b0c is in the history.

Smoke tests read the body now. You were right that a status check proves nothing. Here is the API currently running on the VM answering searchAlbums:

HTTP 200
{"errors":[{"message":"Unknown argument \"input\" on field \"Query.searchAlbums\".","extensions":{"code":"GRAPHQL_VALIDATION_FAILED"}}]}

The old check printed 200 and moved on. The new one splits the status line off the body and rejects the response if it is not 200 or if the body contains an "errors" key. Verified against that same live API: { __typename } passes, searchAlbums is correctly rejected with the body in the error message.

Worth noting this will bite again when #42 lands, since a resolver that throws also returns 200 with an errors array. So the check earns its place beyond this one case.

rachelks

I took another look at the workflow after #42 was merged and noticed a couple of things:

The searchAlbums smoke test is missing the required sort field (AlbumSort!), so the query will fail validation. Adding sort: TITLE_ASC should fix it.
Now that #42 is merged, should we make the workflow fail instead of skipping API deployment when crate-api.service or server/ is missing? Otherwise the deployment job could pass without actually deploying the API.

The smoke tests could also be a bit more robust by validating the JSON response and checking the expected data, rather than just looking for "errors", but I think that's less urgent.

Could you also confirm that the systemd service loads DATABASE_URL from the root .env?

…checks Four points from rachelks on #45. 1. The searchAlbums smoke query omitted `sort`, which is `AlbumSort!` in AlbumSearchInput, so the query failed validation and the check could never pass. Now sends `sort: TITLE_ASC`. 2. The skip guards for crate-api.service and server/ now fail instead of exiting 0. #42 has merged so both are expected, and a deploy job that goes green without deploying anything is the wrong failure mode. 3. The body check parses the JSON rather than grepping for the word "errors". Grep would also fire on an album called "Errors", and it did not verify `data` was present at all. Now rejects a response with errors, with no data, or that is not JSON. Added a totalCount assertion as well, so the check is on the data and not only on its presence. 4. The python validators live in shell variables on one line. A multi-line `python3 -c` heredoc cannot be used inside a YAML block scalar: the block ends at the first unindented line and the workflow stops parsing. Learned by breaking it. Also rebased onto main now that #42 and #43 have merged. Validated the two python expressions against five payloads: a good response, GraphQL errors, null data, a valid totalCount, and two bad ones. Each rejected case prints what was wrong. Co-Authored-By: opencode <noreply@opencode.ai>

erikhfj · 2026-10-15T04:26:55Z

All four, and the first one was a real bug that meant the check could never have passed.

1. sort was missing. Correct. sort is AlbumSort! in AlbumSearchInput, so the query failed validation and the smoke test was dead on arrival. Now sends sort: TITLE_ASC. Worth noting the earlier version of this check was already rejecting that query against the live API and I read the rejection as the check working, when it was the query being wrong. Good catch.

2. Skip guards now fail. You are right that a deploy job going green without deploying is the wrong failure mode, and #42 has merged so both conditions are expected to hold. crate-api.service missing or server/ missing now exit 1 with an error instead of 0.

3. The body check parses JSON now. Grep for the word "errors" had two problems: it would fire on an album called "Errors", and it never verified data was present at all. It now rejects a response that has errors, has data: null, or is not JSON. I also added a totalCount assertion as you suggested, so the check is on the data rather than only on its presence:

{ __typename }           200  ok
searchAlbums             200  ok
totalCount               16 albums
GET /project2/           200  ok

Tested the two validators against five payloads: a good response, GraphQL errors, data: null, a valid totalCount, and two bad ones. Each rejection prints what was wrong.

4. DATABASE_URL and the systemd unit. Yes. The unit has EnvironmentFile=/home/erikhfj/T19-Project-2/.env, that file holds DATABASE_URL=postgres://t19:…@127.0.0.1:5432/crate, and it is gitignored so git checkout --force during a deploy leaves it alone. I verified the md5 is identical before and after a checkout when I set the clone up.

Two things I had to fix along the way, in case they come up again: a multi-line python3 -c heredoc cannot live inside a YAML block scalar, because the block ends at the first unindented line and the workflow stops parsing. And \" inside bash single quotes is a literal backslash, so the python saw payload[\"errors\"] and died on a syntax error. Both are why the validators are now single-line strings in shell variables.

Branch is rebased onto main with #42 and #43 in it.

README was still describing M1, and one line was simply wrong: it said `client/` renders mock data. The mock-data search interface never landed, the client is still the skeleton and #9 wires it straight to GraphQL. - Current milestone moved to M2 - New "Where the code is" table, implemented vs. planned - Local setup for `server/` too: DATABASE_URL, migrate, optional seed - Server commands in the command table - Infrastructure names `crate-api.service` and what CI deploys - M1 row now lists what M1 actually delivered, not mock data that did not land - Sort modes corrected to four, checked against `AlbumSort` in the schema AI usage log gets the eight erikhfj rows missing since 1 October: the #10 identity contract, the #8 VM runbook and the git clone, the #13 facet trade-off, the #43 source-map-js override, the #44 deploy gap, the displayName correction, the #45 review fixes, and this pass. The displayName row is the kind the log intro calls out as most important: the tool drafted a claim that a person had to correct. Per #48 each member documents their own usage, so only erikhfj rows were added. robertky's row from #46 was out of date order at the end of the table and is now with the rest of 9 October, and his relative order within the day is unchanged. Co-Authored-By: opencode <noreply@opencode.ai>

The workflow built server/ in the runner's own workspace and restarted crate-api, but nothing ever put that build where the unit actually runs it. crate-api.service has WorkingDirectory=~/T19-Project-2/server and ExecStart=.../node dist/index.js, so a restart just brought back whatever had been installed on the machine by hand. Reported by rachelks on #42. The deploy step now: 1. checks out $GITHUB_SHA in ~/T19-Project-2, which is a git clone of main as of this issue rather than a hand copy 2. installs and builds server/ there 3. restarts crate-api and polls until /graphql answers 4. runs a smoke suite through the same Apache proxy the browser uses Three things this fixes that were wrong before: - The unit check is [ -f ... ], not `systemctl list-unit-files | grep`. grep -q closes the pipe at the first match, systemctl dies of SIGPIPE, and with pipefail the pipeline reports failure. The if took the else branch and skipped a service that was definitely there. - The restart needs a NOPASSWD sudoers rule, because the runner has no TTY. Scoped to three systemctl commands. - No sleep. Type=simple reports active before Node binds the port, and a measured 1850 ms startup made `sleep 2` produce a 503 through Apache. It polls instead. Verified on the VM: ~/T19-Project-2 is a real clone at 0fbd54a, git fetch works with the deploy key added for that machine, `checkout --force` leaves .gitignore's .env untouched (it holds DATABASE_URL), and sudo is passwordless for the three commands. The skip guard for `server/` stays, because #42 has not merged yet. Once it does, the next push to main will deploy the API end to end. Co-Authored-By: opencode <noreply@opencode.ai>

rachelks: GraphQL returns HTTP 200 even when the payload carries errors, so a status check proves nothing. She is right and it was demonstrable immediately: querying searchAlbums against the API currently running on the VM returns 200 with {"errors":[{"message":"Unknown argument \"input\" on field \"Query.searchAlbums\".","extensions":{"code":"GRAPHQL_VALIDATION_FAILED"}}]} and the old check reported 200 and moved on. The smoke suite now POSTs, splits the status line off the body, and rejects the response if it is not 200 or if the body contains an "errors" key. Verified against the live API: __typename passes, searchAlbums is correctly rejected. Branch also rebased onto main so it carries the source-map-js fix from #43. Reported by rachelks. Co-Authored-By: opencode <noreply@opencode.ai>

…checks Four points from rachelks on #45. 1. The searchAlbums smoke query omitted `sort`, which is `AlbumSort!` in AlbumSearchInput, so the query failed validation and the check could never pass. Now sends `sort: TITLE_ASC`. 2. The skip guards for crate-api.service and server/ now fail instead of exiting 0. #42 has merged so both are expected, and a deploy job that goes green without deploying anything is the wrong failure mode. 3. The body check parses the JSON rather than grepping for the word "errors". Grep would also fire on an album called "Errors", and it did not verify `data` was present at all. Now rejects a response with errors, with no data, or that is not JSON. Added a totalCount assertion as well, so the check is on the data and not only on its presence. 4. The python validators live in shell variables on one line. A multi-line `python3 -c` heredoc cannot be used inside a YAML block scalar: the block ends at the first unindented line and the workflow stops parsing. Learned by breaking it. Also rebased onto main now that #42 and #43 have merged. Validated the two python expressions against five payloads: a good response, GraphQL errors, null data, a valid totalCount, and two bad ones. Each rejected case prints what was wrong. Co-Authored-By: opencode <noreply@opencode.ai>

#46 landed migration 0001 for tracks, simulated users, reviews and crate items, and pointed out that this deploy never runs db:migrate. It was right. A deploy that lands new migration files and does not apply them is the same gap as #44 one layer down: the code is current, the schema is not. Migrate runs before the restart, so the new build is never pointed at an old database. It runs on every deploy because db:migrate is idempotent (verified on #46), not only when the migration list changes. DATABASE_URL is exported from the same .env the systemd unit loads. The server package scripts look for `server/.env` instead, which may not exist in the deploy root, so it is passed explicitly rather than assuming the two agree. Node's --env-file does not override a variable already in the environment, so this wins even when server/.env is present. Verified that a URL-encoded password survives the sed extraction and the handoff to the child process byte for byte. Also rebased onto main, which brings #46 along. Co-Authored-By: opencode <noreply@opencode.ai>

erikhfj · 2026-10-15T19:08:19Z

One more change, and this one is not from review but from #46 merging.

#46 landed migration 0001_user-content-and-tracks.sql for tracks, simulated users, reviews and crate items, and its PR description pointed out that this deploy never runs db:migrate. It was right. The deploy checked out the commit, installed, built and restarted, and never touched the schema. That is the same gap as #44 one layer down: the code is current, the database is not.

db:migrate now runs before the restart, so the new build is never pointed at an old schema. It runs on every deploy rather than only when the migration list changes, because #46 verified it is idempotent.

Two details worth knowing:

Where DATABASE_URL comes from. The systemd unit loads it from ~/T19-Project-2/.env, but the db:migrate script is tsx --env-file-if-exists=.env which resolves against server/. Those are different files and the second may not exist in the deploy root. So the deploy reads DATABASE_URL out of the root .env and passes it explicitly, and fails loudly if it is not there. Node's --env-file does not override a variable already in the environment, so this also wins if server/.env happens to be present.

A password with a URL-encoded character survives the handoff. Checked p%40ss through the sed extraction and into the child process byte for byte. The check that produced a mangled value was my own echo, which does printf-style % expansion in this shell. Worth remembering when reading deploy logs.

Branch is rebased onto main so it carries #46 as well.

Fold #46 into the README and add db:migrate to the #45 deploy, plus the DATABASE_URL reconciliation that came out of it. Co-Authored-By: opencode <noreply@opencode.ai>

rachelks

Looks good now! The migration step is in place and I couldn't find any remaining blocking issues. Thanks for fixing this!

erikhfj requested a review from a team as a code owner October 10, 2026 19:35

erikhfj mentioned this pull request Oct 10, 2026

Deploy the server build to the path crate-api expects #44

Closed

8 tasks

erikhfj force-pushed the feat/44-deploy-server-build branch from e9249e0 to 57db426 Compare October 12, 2026 19:42

rachelks reviewed Oct 13, 2026

View reviewed changes

robertky mentioned this pull request Oct 14, 2026

feat: #5 add user data schema and FTS triggers #46

Merged

rachelks mentioned this pull request Oct 14, 2026

feat: #9 connect client search to GraphQL API #47

Merged

2 tasks

erikhfj force-pushed the feat/44-deploy-server-build branch from 57db426 to 473e4bf Compare October 15, 2026 04:26

rachelks mentioned this pull request Oct 15, 2026

Review and update project documentation for M2 #48

Closed

9 tasks

erikhfj mentioned this pull request Oct 15, 2026

docs: #48 README and AI usage log reflect what has landed #49

Merged

erikhfj and others added 4 commits October 15, 2026 13:05

erikhfj force-pushed the feat/44-deploy-server-build branch from 473e4bf to 4c1ab0d Compare October 15, 2026 19:07

erikhfj added a commit that referenced this pull request Oct 15, 2026

docs: #48 log the #46 follow-up work

8f36881

Fold #46 into the README and add db:migrate to the #45 deploy, plus the DATABASE_URL reconciliation that came out of it. Co-Authored-By: opencode <noreply@opencode.ai>

rachelks approved these changes Oct 15, 2026

View reviewed changes

rachelks merged commit 4d4c9a1 into main Oct 15, 2026
2 checks passed

rachelks deleted the feat/44-deploy-server-build branch October 15, 2026 20:26

evenkkl mentioned this pull request Oct 16, 2026

feat: #7 add batched large catalog seed #50

Merged

Uh oh!

feat: #44 deploy the server build to where crate-api runs it #45

feat: #44 deploy the server build to where crate-api runs it #45

erikhfj commented Oct 10, 2026

Uh oh!

rachelks commented Oct 10, 2026

Uh oh!

erikhfj commented Oct 12, 2026

Uh oh!

rachelks left a comment

Uh oh!

erikhfj commented Oct 15, 2026

Uh oh!

erikhfj commented Oct 15, 2026

Uh oh!

rachelks left a comment

Uh oh!

Uh oh!

Uh oh!

feat: #44 deploy the server build to where crate-api runs it #45

feat: #44 deploy the server build to where crate-api runs it #45

Conversation

erikhfj commented Oct 10, 2026

Why

What this does

The VM work behind it

Three bugs from earlier that this does not repeat

What was tested

What still waits on #42

Closes

Uh oh!

rachelks commented Oct 10, 2026

Uh oh!

erikhfj commented Oct 12, 2026

Uh oh!

rachelks left a comment

Choose a reason for hiding this comment

Uh oh!

erikhfj commented Oct 15, 2026

Uh oh!

erikhfj commented Oct 15, 2026

Uh oh!

rachelks left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!